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Abstract 

Let g > 3 and n > 0, and let M. g , n be the mapping class group of a surface of genus g 
with n boundary components. We prove that M g , n contains a unique subgroup of index 
2 S ~ 1 (2 9 — 1) up to conjugation, a unique subgroup of index 2 9_1 (2 9 + 1) up to conjugation, 
and the other proper subgroups of M. g , n are of index greater than 2 ff_1 (2 9 + 1). In particular, 
the minimum index for a proper subgroup of M g , n is 2 9_1 (2 9 — 1). 

AMS Subject Classification. Primary: 57M99. Secondary: 20G40, 20E28. 

Introduction and statement of results 

The interaction between mapping class groups and finite groups has long been a topic of interest. 
The famous Hurwitz bound of 1893 showed that a closed Riemann surface of genus g has an 
upper bound of 84(g — 1) for the order of its finite subgroups, and Kerckhoff showed that the 
order of finite cyclic subgroups is bounded above by 4g + 2 [18] , [19] . 

The subject of finite index subgroups of mapping class groups was brought into focus by Gross- 
man's discovery that the mapping class group M. g , n = ^o(Homeo(E 5in )) of an oriented surface 
£ Sjn of genus g and n boundary components is residually finite, and thus well-endowed with 
subgroups of finite index [16] . (Homeo(£ 9jn ) denotes the space of those homeomorphisms of 
that preserve the orientation and are the identity on the boundary.) This prompts the 
"dual" question: - 

What is the minimum index mi(.M 9in ) of a pvoper subgroup of finite index in ^Ig^n ? 

Results to date have suggested that, like the maximum finite order question, the minimum 
index question should have an answer that is linear in g. The best previously published bound 
is m\(Aig, n ) > 4g + 4 for g > 3 (see J25j ) . This inequality is used by Aramayona and Souto 
to prove that, if g > 6 and g' < 2g — 1, then any nontrivial homomorphism Mg^n My 



is 



induced by an embedding [lj. It is also an important ingredient in the proof of Zimmermann 
|33| that, for g = 3 and 4, the minimal nontrivial quotient of A4 9j o is Sp2 9 (IF2)- 

The "headline" result of this paper is the following exact, exponential bound. 
Theorem 0.1. For g > 3 and n > 0, 

xm{M g , n ) = mi(Sp 29 (Z)) = mi(Sp 2 ,(F 2 )) = 2^ 1 (2" - 1) . 

This exponential bound is all the more surprising since in similar questions we get linear (ex- 
pected) bounds. For instance, Bridson [61 [7] has proved that a mapping class group of a surface 
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of genus g cannot act by semisimple isometries, without a global fixed point, on a CAT(O) space 
of dimension less than g. The exact minimal dimension for such an action is unknown. On the 
other hand, it has been also shown by Bridson (see [6]) that M g , n has only finitely many irre- 
ducible linear representations over any algebraically closed field, up to dimension (g + 1). Later, 
Funar [15] showed that there is no linear representation with infinite image up to dimension 
about y/g + 1. However, there is an obvious linear representation of rank 2g which comes from 
the action of M. g , n on the homology of E 9j o (the map 9 g ^ n defined below). It is expected that 
this representation is minimal in some sense (see |13j). 

The nontrivial quotient of M g , n of minimal order is unknown, but obviously its order must be 
at least mi(At ff , n ). This quotient is known to be Sp 2ff (F 2 ) if (g,n) = (3,0) or (4,0) (see [33]). A 
consequence of the above theorem is that M. g ^ n > cannot be a quotient of M g , n if 3 < g' < g. 

Our proof of this result is constructive, in ways that we now describe. From the surface we 
obtain a closed oriented surface T, g of genus g by gluing a disk along each boundary component. 
The embedding £ 9>n S 9 induces a first epimorphism M. g , n -M.g,o- The action of Homeo(X 9 ) 
on Hi(E g ) = I? 9 induces a second epimorphism M. g $ -» Sp 2fl (Z) onto the integral symplectic 
group, and, passing mod 2, we obtain a third epimorphism Sp 2fl (Z) -» Sp 29 (F 2 ), where F 2 = 
Z/2Z. From now on we denote by d g ^ n : M. g>n — > Sp 2fl (F 2 ) the composition of these three 
epimorphisms. 

The orthogonal groups 2 4 fl (F 2 ) and O^F^ are subgroups of Sp 2s (F 2 ). The cardinalities of 
SP2g(F2); 0^(F 2 ) and (^(F^ can be found for instance in [30], and from this data it is easily 
shown that, for g > 2, the indices of 0+ (F 2 ) and 0~ g (F 2 ) in Sp 2s (F 2 ) are JV+ = 2 9 ~ l (2 9 + l) and 
N~ = 2 9 ~ 1 (2 9 — 1), respectively. The following, more or less known to experts but seemingly 
unpublished, is the starting-point for our main result (Theorem I0.3|) . 

Theorem 0.2. Let g>3. 

1. 2g (F 2 ) is the unique subgroup o/Sp 2ff (F 2 ) of index N~ , up to conjugation. 

2. 0^(F 2 ) is the unique subgroup o/Sp 2ff (F 2 ) of index N^j" , up to conjugation. 

3. All the other proper subgroups o/Sp 2ff (F 2 ) are of index at least 2N~ . 

We set 0+ n = 6-1(0+ (F 2 )) and 0~ n = e^(0^(¥ 2 )). Thus, by the above, 0~ n is an index 
N~ subgroup of Mg >n , and Og^ n is an index iV+ subgroup of M g , n - Here is our main result. 

Theorem 0.3. Let g > 3 and n > 0. 

1. 0~ n is the unique subgroup of M g>n of index N~ , up to conjugation. 

2. Og^ n is the unique subgroup of M g , n of index Ng, up to conjugation. 

3. If g = 3 then all the other proper subgroups of M.^,n o,re of index strictly greater than 
N+ = 36. 

4- If 9 ^ 4 then all the other proper subgroups of M g , n are of index at least 5N~_ 1 > iV+. 
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Since m is an upper bound for the minimum index of a group G if and only if there is a nontrivial 
homomorphism from G to the symmetric group & m on m letters, one would like to understand 
the permutation representations associated to Theorem 10.31 

We denote by (ft+ n : M g , n — > ®at+ (resp. <p~, n : M- g , n — > ®jv~ ) ^ ne permutation representation 
induced by the action of M. g , n on the right cosets of 0+ n (resp. 0~ n ). Corresponding to the 
numerical relations 

we prove the following. 

Theorem 0.4. 1. Let g > 3 and n > 1. Then (j)~ n : M 9t n ©at- u P to equivalence, 

the unique extension of the representation {<ftg~-i n ) 3 © ^<j~-i n f rom -Mg-i,n to M. g>n , and 
§g,n ■ -Mg^n — > & N + is, up to equivalence, the unique extension of the representation 

0J-l,n © (^s-l,n) 3 f rom Mg-l, n to M g , n - 

2. Let g > 3 and n > 0. Let b be a nonseparating simple closed curve on Xl^ n? and let T^ be 
the Dehn twist around b. Then the cycle structure of the image ofTb under <p~^ n is 

(1) 229_2 (2) 2S ~ 2(29_1 " 1) , 
and the cycle structure of the image of TJ, under <fr+ n is 

(1) 229 ~ 2 (2) 29 ~ 2(2S ~ 1+1 ) 

Remark. Implicit in the statement of Theorem 10.41 is the fact that, if n > 1, then M g -i. n 
naturally embeds into Mg,n- This embedding will be described in Section [2j However, there is 
no natural embedding of A^^-i^ into Mgfl, hence Part (1) of the theorem would make no sense 
for n = 0. 

We observe that the abelianization of M g , n is isomorphic to Z/12 Z if (g, n) = (1, 0), Z n if g = 1 
and n > 1, and Z/10Z if g = 2 (see [20]). Hence, the minimum index of M g , n is 2 if g = 1 or 2. 
Note that M g>n is perfect if g > 3 (see [271 [20]). If = 2 we have N~ = 6 and iV+ = 10, and 
there are six proper subgroups of index at most 10 in M. g>n ' one of index 2, one of index 5, two 
of index 6 and two of index 10. The description of these subgroups as well as the proof of this 
fact are given in Section 

The remainder of the paper is divided into two parts. Part I starts with some preliminaries 
on permutations (Section [1]) and presentations of mapping class groups (Section [2]). Then we 
determine the subgroups of M.2,n of index at most 10 = (Section [3]) and the subgroups of 
M.3 tn of index at most 36 = iV-^" (Section We prove our theorems by induction on the genus. 
The starting case, g = 3, is made in Section 4, and the inductive step is the object of Part II. 
We first treat the case of a surface with a unique boundary component (Sections [5] to [7j) and we 
extend the result to surfaces with several boundary components in Section [51 Theorem 0.1 is 
proved in Section [6l This can be read independently from the rest. 

Acknowledgements. Many thanks to Derek Holt for his helpful advice. The authors are 
pleased to acknowledge the financial support of National University of Singapore research grants 
R-146-000-097-112 and R-146-000-137-112 towards the visits of LP to Singapore in 2007 and 2011 
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supported by the Agence Nationale de la Recherche {pro jet Theorie de Garside, ANR-08-BLAN- 
0269-03). VG is partially supported by the Australian Research Council, grant DP1094072. 

Part I 

Preliminaries 

1 Useful information on permutations 

We define the minimum index mi(G) of a nontrivial group G to be the element of the ordered set 
2<3<4<---<oo corresponding to the minimum among the indices of all proper subgroups 
of G with finite index, and oo when G has no proper finite index subgroup (such a G has been 
called counter- finite) . 

Permutations are important to our investigation because of the well-known relationship between 
index m subgroups and maps to & m . A homomorphism ip : G — > & m is called transitive if its 
image acts transitively on {1, . . . , m}. If ip : G — > & m is transitive, then Stab ¥ ,(l) = {7 G G \ 
<^(7)(1) = 1} is a subgroup of G of index m. Conversely, if H is a subgroup of G of index m, 
then there exists a transitive homomorphism ip : G — > <3 m such that H = Stab^(l) (take the 
action of G on the right cosets of H). It follows that mi(G) is also the smallest m > 2 such that 
there exists a nontrivial homomorphism ip : G — >■ & m . Of course, if such a homomorphism does 
not exist for any m, then mi(G) = 00. 

The minimum index has the property that, if G -» H is an epimorphism, then mi(G) < mi{H). 
(Indeed, the pre-image of an index m subgroup under an epimorphism is an index m subgroup.) 
From the definition of g>n : A4 Si „ — > Sp 29 (F2) as the composition of epimorphisms 

M g , n - M gfi - Sp 29 (Z) -» Sp 29 (F 2 ), 

we therefore have 

mi(7W 9 ,„) < mi(A^ 9 , ) < mi(Sp 29 (Z)) < mi(Sp 29 (F 2 )) . 

Since many generators of mapping class groups commute with each other, we need some pre- 
liminary results that discuss aspects of commuting permutations. In the following, denotes 
the cyclic group of order k, and an orbit of cardinality k under the action of a permutation or 
a permutation group is called a k- orbit. 

Lemma 1.1. Let u G & m have cycle type (l) £l (2)^ 2 • • • {m) lm , and let 

J(u) = {fc€{l,2,...,m}|4>0}, 

so that J2kei(u) ki k = m. 

1. The centralizer C& m {u) is isomorphic to 



kei(u) kei(u) 
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2- If P < C l 6 m (n) is nonabelian, then, for some k G I{u), Ik > mi(P). 
3. If further P is perfect, then P is isomorphic to a subgroup of 



n ((c t )^%)= n VkWtk)- 




If I(P) = {k G I(u) | P acts nontrivially on the union of the k-orbits ofu}, then the 
following numerical constraints hold: 

(a) 5 < mi(P) < whenever k G I(P); 

(b) m > mi(P)(X) fc6 j(p) k); and 

(c) ifm< 5mi(P), then either I{P) = {4} or I(P) - {1} = {2} or {3}. 

Proof. (1) is left to the reader. Let P& denote the projection of P to the component I &t k 
of Ce m (n). Since P is contained in Ilfee/ru) an d P is nonabelian, there exists A; G I{u) such 
that Pfc is nonabelian, and thus it > mi(Pfc) > mi(P). Finally, (3) results from the fact that 
any nontrivial perfect subgroup of (CkY k x &e k (whose support is the union of the fe-orbits of 
u) must have nontrivial perfect image in &e k ■ Then > 5 and the image lies in the maximum 
perfect subgroup 2l£ fe of &£ k . Then (a), (b) and (c) follow readily. □ 

For u G & m we write {l,...,m} = F(u) U S(u) for the partition into the fixed set F(u) 
and support S(u) of the permutation u. Evidently, u restricts to the identity map on F(u), 
respectively to a bijection on S(u). 

Lemma 1.2. Let «,»£ & m be such that uvu = vuv. Then \S(u)\ < 2\S(u) PI 5 , (v)|. 
Proof. If u(i) G P(f), then (vu)(i) = u(i). If also i G F(v), then 



in other words, F(v) n u(F(v)) C P(n). Therefore, 

n n u(P(v)) =0; 

and so S'('u)n'u(P(t;)) C S , (u)nS , (v). However, n maps S(u)nF(?j) bijectively to S'(n) Du(F(v)). 
The result is now immediate from the fact that 



Lemma 1.3. Let u, v G & m be such that uvu = vuv. If k G {2,3} and all nontrivial orbits ofu 
and (u,v) have length k, then u = v. 

Proof. We present the argument for k = 3; for k = 2 it is similar, but simpler. Since u and v 
are conjugate, they have the same cycle decomposition type. If there are no nontrivial orbits, 
then u = v = 1. So, let O be an orbit of u of length 3. Then O must also be a nontrivial orbit 
of (u, v). Hence, since v is a product of 3-cycles, v acts on O as either l,u or -u 2 . In each case, 
the actions of u and v on O commute. Likewise, on each nontrivial orbit of v the actions of u 
and v commute. Finally, on F(u) PI P(t> ), since u and u both act as the identity, the actions of 
u and v again commute. Hence, uv = vu. From uvu = vuv, the result follows. □ 



u(u(i)) = (uvu)(i) = (vuv)(i) = (vu)(i) = u(i) . 



Thus, 





\s( u )\ = \s(u) n s(v)\ + \s(u) n f( v )\ . □ 
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Lemma 1.4. Let u,vq,vi £<5 m be such that 

(a) uv{U = ViUVi for i = 0, 1; and 

(b) vq and v% commute. 

If u has order 3 and all nontrivial orbits of (u,vo) and (u,vi) have length 4, then vq = v±. 

Proof. If u is trivial then the result is immediate from (a). Therefore, we can assume that u 
contains the 3-cycle (1 2 3) and {2, 3,4} is a nontrivial orbit of vq. Now, in 64 

(1 2 3) (4 3 2)(1 2 3) = (4 3 2)(1 2 3) (4 3 2) = (1 4) (2 3) ; 

however 

(1 2 3) (4 2 3)(1 2 3) / (4 2 3)(1 2 3) (4 2 3) . 
Thus, vo contains the cycle (4 3 2) in its cycle decomposition. Because v% commutes with vq, 

(1.1) «i(4 3 2)v^ = ( Vl (4) «i(3) ui(2)) 

is a 3-cycle in the decomposition of vo. Now, (u, v\) has a 4-orbit of the form {1, 2, 3, x], whence, 
from the above argument in ©4, v\ acts on this orbit as (3)(x 2 1), (2)(x 1 3) or (l)(x 3 2). 
However, in the first case, because fi(3) = 3, Equality (jl.ip implies that i>i(2) = 2, contradicting 
i>i(2) = 1. Similarly, in the second case, v\(2) = 2 combines with (jl.ip to imply that t>i(3) = 3, 
contradicting «i(l) = 3. 

This leaves the last case, in which i>i(3) = 2, which from (jl.ip gives x = i>i(2) = 4. That is, 
(4 3 2) is a 3-cycle in the decomposition of v\. It follows that the 3-cycles of vq and vi coincide, 
whence Vo = v%. □ 

Lemma 1.5. Let u,Vq,v\,V2 G 6 m be such that 

(a) uviU = ViUVi for i = 0, 1, 2; 

f&j Wo j y i an ^ w 2 commute pairwise; and 

(c) whenever {i,j,k} = {0,1,2}, there is an isomorphism 7,-^ from (u,Vi,Vj) to (u,Vi,Vk) 
fixing u and vi, and sending Vj to v^. 

If u is a product of 4- cycles (possibly with fixed points) and all nontrivial orbits of each {u,Vi) 
have length 4, then vq = v\ = vi- 

Proof. Since, by (a), u and V{ are conjugate, they have the same cycle decomposition type, 
which must be a product of 4-cycles. If there are no nontrivial orbits, then each Vi = 1. 

Let O be an orbit of u of length 4. Then O must also be a nontrivial orbit of each (u, Vi). From 
(a) it follows that O is also a nontrivial orbit of each v^. Likewise, a nontrivial orbit of any 
one Vi must also be a nontrivial orbit of u and thereby of each «j. Thus, it suffices to restrict 
attention to the action on each nontrivial orbit O; in effect, (u, Vo, vi, V2) < 64- Using (b), from 
the fact that in 64 commuting 4-cycles are either the same or mutually inverse, we must have 
at least two distinct indices i,j G {0, 1, 2} such that vi = Vj. Since by (c) 7^ both fixes Vi and 
sends Vj to Vk, we conclude that Vi = Vj = Vk- □ 
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2 Presentation for the mapping class group 

Throughout the paper we denote by TJ, the Dehn twist about a simple closed curve b. We 
fix a representation of £ 9i i as well as the simple closed curves Oq, ai, . . . , a 2g +i illustrated in 
Figure 2.1, and we set T\ = T ai for all < i < 2g + 1. The following result is shown in |23j . 




Theorem 2.1. (Matsumoto [23]). Let g > 2. 

.M Sj i /ias a presentation with generators Tq,T\, . . . ,T 2g and relations 

TiTjTi = TjTiTj if a, and aj intersect in a single point, 
TiTj = TjTi if a* D Oj = , 
(T 2 T 3 T 4 T ) W = (r 1 T 2 r 3 r4To) 6 , 
(T 2 T 3 T 4 T 5 T 6 T ) 12 = {T^T^T^nf if g > 3 . 

b) Mgfi is the quotient of Ai 9t i by the additional relation 

T^- 2 = (T T 3 T 4 ---T 2g ^-\ 

□ 

We call k simple closed curves bi,...,bk in a k- chain if their intersection numbers satisfy 
i(bi,bj) = 1 if \i — j\ = 1 and i(bi,bj) = otherwise. Such a fc-chain is called nonseparating if 
the complement of b± U • • • U bk in is connected. Note that, if (bi, . . . , bk) is a nonseparating 
/c-chain, then each 6j is nonseparating, too. The next lemma follows from the rigidity of closed 
Riemann surfaces (see, for example, |14| Sections 1.3 and 2.3). 

Lemma 2.2. Let ...,6fe) and (b[, . . . , b' k ) be two nonseparating k-chains in T, g>n . Then 
there exists a £ Mg,n such that a{b{) = b^ for all 1 < i < k. In consequence, this a satisfies 
aT^a^ 1 = Ty for all 1 < i < k. □ 

For studying the mapping class group M g , n with n > 2 we use the following convention: 
ao, a 2 , ■ ■ ■ , a 2g , a 2g+ \ and b\, . . . , b n are the simple closed curves illustrated in Figure 2.2, Ti = T ai 
for all < i < 2g + 1, i ^ 1, and Tj = Tf,. for all 1 < j < n. In order to unify statements for 
n = 1 and n > 1, we make the further convention that, when n = 1, b\ in Figure 2.2 coincides 
with a\ in Figure 2.1. Then for n = 1 the element T{ is simply T\. 

The following is well-known. (It can be found for instance in [21, Prop. 2.10 and Thm 3.1]. 
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Figure 2.2. Generators for M g - 



Proposition 2.3. Let g > 2 and n > 1. T/ien -M 9;n is generated by To, T2, . . . , T2 9 , T{, . . . , T' n . 

□ 

Observation. Let g > 3 and n > 1. There is an injective homomorphism Ai g ~i >n — > M g , n 
which sends T- to T- and T to T for all j G {1, . . . , n} and i G {0, 2, . . . , 2g — 2}. It is easily 
seen that this homomorphism is induced by some embedding of £ 9 _i jn into £g,n- From now on 
we will assume M. g ~i t n to be embedded into M. g , n via this homomorphism. Note that such a 
homomorphism does not exist for n = 0. 

3 The genus 2 case 

In this section we describe all the subgroups of M,2,n of index at most iV 2 + = 10 up to conjugation. 
We will see in particular that the genus g = 2 case is different from the genus g > 3 case. 

The first difference comes from the fact that the abelianization of M.i,n is nontrivial, while the 
group M. g , n is perfect if g > 3 (see [27], [20]). More precisely, the abelianization of M.2,n is 
Z/10Z |24j . So, if ab : M.2,n Z/10Z denotes the projection of 7W2,n. onto its abelianization, 
then ab _1 (Z/5Z) is a subgroup of index 2, ab _1 (Z/2Z) is a subgroup of index 5, and Ker(ab) 
is a subgroup of index 10. Note that all these subgroups contain the commutator subgroup 
M-2 n - In particular, since Sp 4 (F2)' has index 2 in Sp 4 (F2), neither ab _1 (Z/2Z) nor Ker(ab) is 
a pre- image under #2,n of a subgroup of Sp 4 (F2). 

The second difference comes from the fact that Sp 4 (F2) = ©6 [11] contains more than two 
subgroups of index at most 10, up to conjugation. In addition to 07 (F2) = ©5 [H] and 4 (F2), 
of indices 6 and 10 respectively, it contains the alternating group 2lg of index 2, and another 
subgroup of index 6 which can be described as follows. The group Sp 4 (F2) = ©6 has a non-inner 
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automorphism a denned by 

(1 2) ^ (1 2)(3 5)(4 6) 
(2 3) ' — y (1 3)(2 4)(5 6) 
n : < (3 4) ^ (1 2) (3 6) (4 5) 
(4 5)^(1 3) (2 5) (4 6) 
(5 6) ' — y (1 2) (3 4) (5 6) 

(Since Out(©6) has order 2, a is essentially unique.) It turns out that a(OJ (F2)) is a subgroup 
of Sp 4 (F2) of index 6 which is not conjugate to Oj(F2). On the other hand, a(0^ (F2)) is 
conjugate to (F2) (so there are four subgroups and not five), and ab _1 (Z/5Z) = O^tfUe). 
Note also that Out((3 n ) is trivial if n 7^ 6. 

So, we have the following subgroups of index at most 10 in M-2, n up to conjugation: 

• ^(Sle) = ab _1 (Z/5Z) of index 2, 

• ab- 1 (Z/2Z) of index 5, 

• °2,n(°4 ( F 2)) of index 6, 

• ^i(a(Or(F 2 ))) of index 6, 

• Ker(ab) of index 10, and 

• 02,n(°t ( F 2)) of index 10. 

We show that these are all the proper subgroups of index at most 10 in M.2,n up to conjugation. 

Let n > 1, m > 1 and u> G <5 m . If u> 10 = 1, then there is a permutation representation 
cycly, : M-2,n — > &m which sends Tj and Tj to w for all i G {0,2,3,4} and all j G {1, . . . , n}. 



Such a representation is called a cyclic representation of .M2,; 
is a cycle of length m and m G {1, 2, 5, 10}. 



It is transitive if and only if u> 



Lemma 3.1. For n > 1, there are exactly three conjugacy classes of non-cyclic transitive permu- 
tation representations of M.2,n of degree at most 10, namely, two conjugacy classes of degree Q, 
and a unique conjugacy class of degree 10. More specifically, up to conjugacy in &§, the two 
transitive permutation representations M.2n — > ©6 are given by 



: < 



: < 





(1 2) 


/or 1 < 


j < n 




(2 3; 






T 3 ' y 


(3 4; 






T 4 ^ 


(4 5; 








(5 6; 








(1 2) 


(3 5)(4 6) 


for 1 < j < n 


T 2 ' y 


(1 3; 


(2 4) (5 6) 




T 3 ^ 


(1 2; 


(3 6) (4 5) 




T 4 ^ 


(1 3; 


(2 5) (4 6) 




T ^ 


(1 2; 


(3 4) (5 6) 
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and, up to conjugacy in &iq, the unique permutation representation M2, n — > &io is given by 

Tj i-> (3 5)(6 8)(9 10) forl<j<n 
T 2 ' ^ (2 3) (4 6) (7 9) 
,,.„ : \ T 3 ^ (1 2)(6 10)(8 9) 
T 4 t-> (2 4) (3 6) (5 8) 
T i-> (4 7) (6 9) (8 10) 

Proof. In the case n = 1, this result can be easily proved with a direct calculation. Using the 
presentation of M% i from Theorem l2.ll one can use coset enumeration techniques to perform a 
systematic search for representatives of the conjugacy classes of subgroups of M 2 ,i of index at 
most K for a (small) integer K; see |29j . When .K" = 10, this systematic search shows that there 
are exactly six conjugacy classes of proper subgroups of M.2,1 of index at most 10; the columns 
of the coset table for the each of the constructed subgroups yield the images of the generators 
To, . . . ,T4 under the corresponding permutation representation. The computation is very easy 
and can be performed with any mathematical software such as Magma or GAP. 
Now suppose that n > 2. Let ip : M2, n &m be a non-cyclic and transitive representation with 
m < 10. For 1 < j < n, we denote by M^' the subgroup of M.2,n generated by Tj, T2, T3, T4, To. 
This group is isomorphic to M.2,1 via an isomorphism jj : M.2,1 — ► M^ which sends T\ to Tj 
and T to Tj for all i G {2, 3, 4, 0}. We denote by cpj : .A/f 2,1 — > &m the composition of 7^ with <p. 
Observe that 

(*) <Pj(Ti) =<p(Ti) for allj E {1, . . . ,n} and i € {0,2,3,4}. 

By the case n = 1, for each j G {1, . . . , n} there is a decomposition {1, . . . , m} = 5- US'j 2 ' 1 US'- 
as follows. Each is invariant under the action of ifj(M2,i)', either Sj 1 ^ = 0, or ipj restricted 

to Sj is equivalent to an element of {02^i>0i>02^i}j ^ "Sj * s nonempty, then there exists 

(2) (2) 
wj G 6 m — {1} such that S(wj) = Sj and the restriction of tpj to Sj is cycl^.; and tpj(M2,i) 

acts trivially on •Sj 3 ''. 

Let 4> G {0^i> 01 ; <^il and set N = N 4> = 6 if <t> = or <j%, and TV = = 10 if <f> = (f>^ v 
The following claims are readily verified from the description of <fi, using either GAP or Magma 
where necessary: 

1. <t>(T 2 ) and (^(T^) have no common cycle in their decompositions; 

2. S({(j)(Ti) I % = 1,3,4,0}) = {1,...,N}, where, for IC6jv, S(X) denotes \J weX S(w); 

3. the (simultaneous) centralizer of {(/>(T) | i = 1,3,4,0} in & n is {1, <p(Ti)}; 

4. the support of each cycle in the decomposition of 4>(Ti) intersects 5({0(T) | i = 0, 2, 3, 4}) 
nontrivially. 

We first show that each Sj = 0. Whenever Sj / 0, then by (1) we get that u;.,- is the product 

of the common nontrivial cycles of (pj(T2) and (pj(T 3 ). However, by (*), these common cycles 

(2) 

are independent of choice of j. It follows that for all j G {1, ... ,n}, Sj / 0, t«j = w\ and 

sj 2 ^ = S(uij) = S(w\) = S± . However, ip is transitive and non-cyclic. Hence, sj = for all 
i E {1, . . . , n}. 
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It now follows that each S-^% } and therefore, the restriction of each ipj to Sj is equivalent 

to an element of {<f>2 i> 4>i > ^21 }■ Since we know by (*) that for each i 1 the ipj(Ti) agree, it 

(3) 

remains to show that all the ipj(T\) also coincide, and that each S)j is empty. 

Without loss of generality we can assume that = {1, . . . , N} and the restriction of ip% to 
{1,...,N} is an element <j> in {$21, $\i $2 iIj where N = N^. Let j G {l,...,n}. Since Tj 
commutes with T{, T3, T4, To, the permutation ipj(T\) belongs to the centralizer of {(/?i(Tj) | i = 
1,3,4,0}. Combining (2) and (3) we get that this centralizer is {l,<^(Xi)} x 6 m _jy On the 
other hand, by (4), the support of each cycle of (fj(Ti) intersects S({(fj(Ti) \ i = 0,2,3,4}) 
nontrivially, and, by (*), 

S({^(T t ) I i = 0,2,3,4}) =S({<p 1 (T i ) |i = 0,2,3,4}) C{1,...,N}. 

This implies that ipj(T\) = 4>{T\) and, therefore, each ipj(Ti) is the same. Then, finally, by 
transitivity, it must be that m = N, and the proof is complete. □ 

Proposition 3.2. Let n > 0. Then M.2n has precisely six proper subgroups of index at most 
10 up to conjugation, namely, ab _1 (Z/5Z) = #^T n (2l6) of index 2, ab _1 (Z/2Z) of index 5, 
6 2,n(°i ( F a)) of index 6, 6>^(a(07 (F 2 ))) of index 6, Ker(ab) of index 10, and 6>^(Oj(F 2 )) 
of index 10. 

Proof. For n > 1, the claim is a direct consequence of Lemma 13. II together with the description 
of the subgroups of M.2,n given in the beginning of the section. By Theorem 12.11 M.2,0 is a 
quotient of by one additional relation, so for the case n = it is sufficient to check that 

the representations of M.2,1 given in Lemma 13.11 satisfy the additional relation of A^2,oj this is 
the case, as can easily be verified. □ 



4 The genus 3 case 



In this section we calculate the subgroups of index at most 36 in M^ ;n U P to conjugation. 
(Note that N§ = 28 and = 36.) We argue in the same manner as for the case of genus 2 
surfaces (see Section[3]), with direct calculations often made with computers. However, we should 
point out here that, in this case, the computations are far from being elementary, and we often 
approach the limit of what can currently be done with computers (especially in the proof of 
Lemma |4.1|) . Recall also that the case of surfaces of genus g = 3 will be the first step in the 
induction to prove Theorem 10.31 



Lemma 4.1. For n > 1, there are exactly two conjugacy classes of nontrivial transitive permu- 
tation representations of Ai-^^ n of degree at most 36, namely a unique conjugacy class of degree 28 
and a unique conjugacy class of degree 36. More specifically, up to conjugacy in S28, the unique 
permutation representation -M3 jn — > &28 is given by 

Tj h-> (14 18) (16 21) (17 22) (19 23) (24 26) (27 28) for 1 < j < n 
T ^ (1 3) (2 5) (4 8) (20 25) (24 28) (26 27) 
T 2 ^ (10 14)(12 16)(13 17)(15 19)(20 24)(25 28) 
>-, M . s, T 3 ^ (6 10) (7 12) (9 13) (11 15) (24 27) (26 28) 
T 4 ^ (3 6) (4 7) (5 9) (15 20) (19 24) (23 26) 
T 5 ^ (2 4) (5 8)(6 11)(10 15)(14 19)(18 23) 
T 6 ^ (1 2)(3 5) (6 9)(10 13)(14 17)(18 22) 



Finite index subgroups of mapping class groups January 20, 2013 



12 



and, up to conjugacy in the unique permutation representation M^^ n —> S36 is given by 

' Tj (6 9)(10 13)(14 18)(15 19)(20 22)(21 25)(26 28)(27 30)(31 32)(34 35) 

for 1 < j < n 

T ■-> (1 2)(11 17)(14 22)(16 24)(18 20)(21 28)(23 29)(25 26)(27 32)(30 31) 
+ I T 2 H- (4 6)(7 10)(11 14)(12 15)(16 21)(17 22)(23 27)(24 28)(29 32)(33 35) 
^3,n \ T 3 ^ (3 4)(5 7)(8 12)(14 20)(18 22)(21 26)(25 28)(27 31)(30 32)(33 36) 

T 4 ^ (2 3)(7 H)(10 14)(12 16)(13 18)(15 21)(19 25)(29 33)(31 34)(32 35) 
T 5 ^ (3 5)(4 7) (6 10)(9 13)(16 23)(21 27)(24 29)(25 30)(26 31)(28 32) 
k T 6 i-> (5 8)(7 12)(10 15)(11 16)(13 19)(14 21)(17 24)(18 25)(20 26)(22 28) 

In particular, mi(.A/f3 >n ) = 28. 

Proof. In the case n = 1, the result is shown by a direct computation. Using the presentation 
of M.31 from Theorem 12.11 one can use coset enumeration techniques to perform a systematic 
search for representatives of the conjugacy classes of subgroups of of index at most K for 
a (small) integer K; see [29] . When K = 36, this systematic search shows that there are exactly 
two conjugacy classes of proper subgroups of M.^,\ of index at most 36: exactly one conjugacy 
class of subgroups of index 28 and exactly one conjugacy class of subgroups of index 36. The 
columns of the coset table for the each of the constructed subgroups yield the images of the 
generators Tq,...,Tq under the corresponding permutation representations ^ 3 ~ 1 of degree 28 
and 2 of degree 36. 

We used the implementation of the low index subgroup search provided in Magma [5j, filling 
the coset table in column major order. We ran a development version of Magma V2.15. The 
computation took approximately 47.5 hours on a GNU / Linux system with an Intel E8400 64-bit 
CPU (core: 3 GHz, FSB: 1333 MHz) and a main memory bandwidth of 6.5GB/s (X38 chipset, 
dual channel DDR2 RAM, memory bus: 1066 MHz). We remark that the use of column major 
order is crucial for the running time; tests for indices between 10 and 15 suggest a speed-up by 
a factor between 10 3 and 10 4 compared to row major order. 

Now suppose that n > 2. Let <p : M^,n — > & m be a transitive representation with m < 36. For 
1 < J ' < n, we denote by M.^> the subgroup of M.^^ n generated by Tj, T2, . . . , Tq, Tq. This group 
is isomorphic to via an isomorphism jj : — > A4^' which sends T\ to Tj and Tj to T,, 

for all i € {2, ... , 6, 0}. We denote by ipj : ^3,1 — > & m the composition of jj with ip. Observe 
that 

(*) <pj(Ti) = (p(Ti) for all j G {1,... ,n} and i G {0,2,... ,6}. 

By the case n = 1, for each j G {1, ...,n} there is a decomposition {l,...,m} = U 

Sj 2 ^ as follows. Each Sj k ^ is invariant under the action of yj(A^3,i); the restriction of (fj to 

Sj is equivalent to an element of {0 3 ~ 1 ,0 3 f 1 }; and ^(^3,1) acts trivially on <sj 2 \ (Note 
here a difference from the g = 2 case: because is a perfect group [27] it has no cyclic 

represent at ions . ) 

Let 4> G {4>3i-,4>ti} an d set N = = 28 if (f> = 4>^ 1 , and N = = 36 if (j) = Again, 
the following claims may be confirmed from the description of <p, using either GAP or Magma 
where necessary: 

1. S({<j>{Ti) I i = 1,3,4,5,6,0}) = {1,. . .,N}, where, for X QS N , S(X) denotes U weX S(w); 
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2. the unique element of the centralizer of {(f>(Ti) \ i = 1,3,4,5,6,0} in &n having the same 
cycle decomposition type as 4>(Ti) is 4>(Ti); 

3. the support of each nontrivial cycle in the decomposition of <p(Ti) intersects S({4>(Ti) \ i = 
0,2,3,4,5,6}) nontrivially. 

Without loss of generality we can assume that = {1, . . . , N} and the restriction of if\ to 
{1, . . . , N} is an element (ft m {03~i> fit ll> wnere N = N^. Let j G {1, . . . , n}. Since T- commutes 
with T[, T3, T4, T5, Tq, To, the permutation tfj(Ti) belongs to the centralizer of {ipi(T{) \ i = 
1,3,4,5,6,0}. By (1), this centralizer is Z x 6 m _Ar, where Z is the centralizer of {<j)(Ti) \ 
i = 1,3,4,5,6,0} in &n- By (3), the support of each nontrivial cycle of ^Pj{T\) intersects 
S({(fj(Ti) I i = 0,2,3,4,5,6}) nontrivially, and, by (*), 

S({^-(Ti) I i = 0,2,3,4,5,6}) 
=S({i Pl (T t ) I i = 0,2,3,4,5,6}) 
C{l,...,iV}. 

Thus, (fj(Ti) £ Z. Now, since Tj and T[ are conjugate in A^3,i, tpj{T\) and ^i(Ti) share the 
same cycle decomposition type; hence, by (2), tpj(Ti) = (p\{Ti). To complete the proof, observe 
that transitivity forces m = N. □ 

Proposition 4.2. Let n > 0. Then Ms t n has precisely two proper subgroups of index at most 
36 up to conjugation, namely, 0^ n of index 28, and 0^ n of index 36. 

Proof. For n > 1, the claim is a direct consequence of Lemma 14.11 and the definitions of 0% n 
respectively 0% n . By Theorem 12.11 A^o is a quotient of Mz,i by one additional relation, so 
for the case n = it is sufficient to check that the representations of given in Lemma 14. II 
satisfy the additional relation of A^3 5 o; this is the case, as can easily be verified. □ 

Part II 

Induction arguments 

We turn now to the proof of our main result, Theorem 10. 3[ As pointed out before, we argue by 
induction on the genus. Recall that the case g = 3 is proved in Section H] (see Proposition 14. 2p . 
Thus: 

• from now on, we suppose that g > 4 plus the inductive hypothesis that Theorem \0.3\ holds 
for a surface of genus g — 1 . 

Recall that we have defined N~ = 2^(29 - 1) and iV+ = 2^- 1 (2f + 1). Throughout the 
arguments below, we shall rely on the following numerical relationships for g > 4. 

^+ = 3Ar+_ 1 + AT-_ 1 , N- = 3N-_ 1 + N+_ lt 
4N g _, < Ng < N+ < 5JV-_! < 2N~ . 

(Actually, only the third inequality requires g > 4; the others hold for g > 2.) 

Theorem 10.21 will be entirely proved in Section Theorem 10.41 will be proved in Sections [3 

and El 
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5 Factorization through symplectic groups 

Our goal in this section is to prove the following theorem. 

Theorem 5.1. For g > 4, let if : M g ,i — > S m be a nontrivial homomorphism, with m < 5N g _ 1 . 
Then there is a homomorphism <p : Sp 2g (^2) — > 6 m such that the following diagram commutes. 




The proof strategy is as follows. To show that the kernel of <p contains the kernel of Og i, 
we first prove that the image Wi = <p(Ti) has order 2 in <5 m . For this, we consider the cycle 
decomposition type (1)^ 1 (2)^ 2 • • • (m)^ m of the permutation W{. We first exclude the possibility 
of cycles of length at least 5, then of cycles of length 4, and finally - the most delicate case - 
we exclude cycles of length 3. Hence, Wi is reduced to being an involution after all. It then 
remains to show that the kernel of 9 g i is the normal closure of the square of any of our standard 
generators of M. 9j \. 

5.1 Reduction to involutions 

Proposition 5.2. Let g > A, and let b be a nonseparating simple closed curve in S g i. Then 
the centralizer Z^ oj'T^ in M g> i contains an index 2 subgroup Z^ with the properties: 

(a) Z^ is perfect; 

(b) N g _, < mi(Z+); and 

(c) T b G Z+. 

Proof. It is known that Zf, is the set of mapping classes that fix the curve b up to isotopy (see 
e.g. We take Z^ to be the subgroup of Z^ consisting of those classes that also preserve 

the orientation of b. Evidently, Z^~ has index 2 in Z\, and contains From [26], Z^ is the 
image of M. g -\^ in M. g ^\ under the homomorphism induced by a quotient map £ 9 -i,3 — > E 9) i 
identifying two boundary circles with the closed curve b. Since by [IT] Ai g -i t s is perfect, we 
have (a). From the epimorphism ftA g ~i y 3 — > Z^~ we know that mi(.M 9 -i > 3) < m K^b~)^ and by 
induction we have m\(M. g _i^) = thus N g '_ 1 < mi(Z^). □ 

As ever, we consider the simple closed curves ao, a±, . . . , a 2g +i illustrated in Figure 2.1, we 
denote by Tj the Dehn twist about a^, and write Wi for the image under (p of the Dehn twist Tj 
(i = 0, 1, . . . , 2<; + l). In the sequel we repeatedly use the fact that, since Tj and Tj are conjugate 
in the mapping class group, Wi = </?(Tj) and Wj = f(Tj) are conjugate in the symmetric group, 
and so share the same cycle decomposition type, say (1)^ 1 (2)^ 2 • • • (m) m , where Y^/k=i = m - 
The fact that Tj G Z^. implies that, whenever l\. > with k > 1, ip(Z^.) acts nontrivially on the 
union of the /c-orbits of W{. Therefore, the above proposition combines with Lemma fl.ll f3) (a), (c) 
to yield the following. 
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Corollary 5.3. For g > 4, let tp : -M g ,i —> 6 m be a nontrivial homomorphism, with m < 5N g _ 1 . 
Then there exists k € {2, 3, 4} such that every W{ has the same cycle decomposition type (l)^ 1 (kY k 
with£k>N~_ v □ 

Lemma 5.4. For g > 3, let tp : M g> \ — > & m be a group homomorphism. If S(w 2g - 2 ) ^S(w 2g ) = 
0, then cp is trivial. 



Proof. First note that, since conjugate permutations have bijective supports, by Lemma [2721 
the cardinality r = |S*(tt;j)| is independent of choice of i, and for all 3-chains (oj,Oj,Ofc) yields 
the same cardinalities s = \S(wi) n S(wj)\ and t = \S(wi) H S(wk)\- By assumption, the 3-chain 
(029-2,029-1,029) gives t = 0, which implies that 

(£(1029-2) n S(w 2 g-i)) u (S(w 2g -i) n S(w 2g )) c S(w 2g -i) , 

and so r > 2s. On the other hand, Lemma 11.21 asserts that r < 2s. Thus, for any 3-chain 

(aj, dj, dfc) we have 

S( Wj ) = (S(wi) n S( Wj )) u (S( Wj ) n S(w k )) . 

Turning now to the 3-chains (00,04,03) and (00,04,05) and (03,04,05), we observe that, since 
t = 0, the subsets S(wo) D S(w4), S(ws) n S(w4) and £(104) H £(105) of S(w^) are pairwise 
disjoint, but each of cardinality half that of S(w4). For avoidance of contradiction, it must be 
that r = s = 0; whence, since by Theorem 12.11 the image of tp is generated by the Wi, (p is 
trivial. □ 

Proposition 5.5. For g > 4, let tp : M g ,\ — > S m iwi/i m < 5iV7/_ 1 . Then for i = 0, 1, . . . , 2g + 1 

i/iere are no ^-cycles in the cycle decomposition ofwi. 

Proof. Again write Ik for the number of /c-cycles in the decomposition of w 2g , and suppose 
that £4 > 0. Then Corollary 15.31 forces the cycle decomposition type of every Wj to be (1)^(4)^ 4 
where £4 > N~_ v The image tp(M g< i) contains the subgroup H = (w 2g , w 2g+ i), which, since 
w 2g is a product of 4-cycles, is a homomorphic image (via X{ h-> w 2g+ i) of the group 

H = (xq,xi I = xf = 1, a^o^i^o = xiXqXi) 
of order 96 (denoted (—2, 3 | 4) in [10, p. 74]). 

From now on M. g -i t \ is considered as the subgroup of M g ,i generated by To, T\, . . . , T 2g - 2 - We 
write fifc for the union of the orbits Qk,i (i = 1, • • • , hk) of cardinality k of H. Thus, \£lk\ = khk 
and k divides 96. From the fact that the generators w 2g and W29+1 of H contain no nontrivial 
cycles of length less than 4 it follows that h 2 = /13 = 0. Since tp(M g — 1 1) lies in the centralizer of 
H, it acts, for any k, both on and on the set {£lk,i \ i = 1, • • • , hk}. The former corresponds to 
a homomorphism ipf. : M. g -i ; i — > &kh k and the latter to a homomorphism Vk ■ M g ~i,i — > &h h - 

Claim. If Vk i> s trivial, then so is ipk- 

To prove this, since v\ = ip\ we may assume that k > 4. Assuming that z^. is trivial, we observe 
that each fc-orbit £lk.i of i/ must be invariant under the action of ^(^9-1,1). Now partition 
Qk,i by means of the orbits of ^29, as 

Qfc = Vi U • • • U Vp U F , 
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where each orbit Vj of W2 g has length 4, and F consists of the fixed points of wi g . Then 
p(M g -i t i) acts on V\ U • • • \JV P , on {Vi, • • • ,V P }, and on F. The action on the set {V\, . . . ,V P } 
must be trivial because 

k Q6 

P < 4 < T < 2*- 2 (2*- 1 - 1) = N~_i = miiM^x) 

(by induction). This leaves (p^A/lg—ii) acting on each 4-orbit "Pj, where the action must again 
be trivial, since certainly 4 < mi(.M s _i j i). Hence, the action of tp(A4 g -i t i) on Qk,i stabilises 
^k,i H S(w2 g ) pointwise, and likewise n S^u^g+i)- However, since H is generated by W2 g 
and W2g+i; the union of n S{w2 g ) and $ n S^w^+i) is equal to O^j. Thus, the action of 
ip(M g —\ i) is trivial on each fifc 5 j, and therefore also on their union fi^ as required. 

In consequence, if ip(Ai g -i i) acts nontrivially on some Ojt, then /i^, > mi(.Mg-i i). Since 

A;/ifc = < m < 5 A r 9 l 1 = 5mi(A^ g _i i i) , 

the only possibilities are k = 1 or = 4. From Lemma 15.41 .M ff -i i must act nontrivially on 
some with > 1, and therefore k = 4 indeed occurs. 

The inequality h$ > implies that h\ < m — 4/14 < N~ 1} thus acts trivially on S7i . 

It follows that 

%(M r i,i))cn 4 cS(ff), 

where, for a subgroup G of 6 m , 5(G) denotes its support. Since S(ip(M. g -\ j i)) contains 
S((wi, W2}), which by Lemma I2T21 has the same cardinality as S(H), the above inclusions are 
indeed equalities. 

So, W2 g is a product of 4-cycles and all nontrivial orbits of H = {w2 g ,W2 g +i) have length 4. 
Applying again Lemma \2.2\ we get that u>4 is a product of 4-cycles and, for i 6 {0,3,5}, all 
nontrivial orbits of (w^Wj) have length 4. By Lemma [T31 we deduce that wq = W3 = w&. But, 
since W3 = w§, we also have 

w 3 = (w^^wsiw^y 1 = (w 5 w e )w 5 (w 5 w e )~ 1 = Wq. 

Thus W5 = wq, and therefore wq = w\ = • • • = W2 g - It follows that the image of ip is cyclic, and 
hence, because M. g ,\ is perfect, tp is trivial - a contradiction. □ 

Proposition 5.6. For g > 4, let tp : M. g ,i — » & m with m < 5N g _ 1 . Then for < % < 2g + 1 
there are no 3-cycles in the cycle decomposition ofwi. 

Proof. Suppose to the contrary that £3 > 0. In particular, ip cannot be trivial. Then Corol- 
lary [53] forces the cycle decomposition type of each W{ to be (1) x (3) 3 where £3 > N~_ v The 
image <p(M. g> i) contains the subgroup H = (u>2 9 , u>2 9 +i), which, since W2 g is a product of 3- 
cycles, is a homomorphic image (via x\ 1— > W2 g +i) of the group 

H = (xq,Xi I Xq = x\ = 1, XqX\Xq = XlXQXl) 

of order 24. Again, we consider At ff _i,i as the subgroup of M. g> i generated by To, T\, . . . , T^g-i- 
Moreover, we write ilk for the union of the orbits Qk,i (i = 1, • • • ,hk) of cardinality k of H. 
Thus, = kh], and k divides 24. From the fact that the generators W2 g and W2 g +i of H 
contain no nontrivial cycles of length less than 3 it follows that hi = 0. 
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Since ip(M g -\ i) lies in the centralizer of H, it acts, for any k, both on £1^ and on the set 
{Clk,i I i = 1) • • • > hk}- The former corresponds to a homomorphism i/j^ : M g -\ t i — > ©fc/i fe and 
the latter to a homomorphism : A^ fl — 1 1 —5- <5/i fe - 

Claim 1. If Vk is trivial, then so is ip}.. 

Indeed, suppose that V}~ is trivial. Then each fc-orbit f2fc j of H must be invariant under the 
action of (p(M. g -i t i). But 

|n fc)i | = k < 24 < 23-\2V- 1 - 1) = mi(M g -i,i) , 

thus (p(Adg-i t i) acts trivially on each VLk,i, that is, acts trivially on VL^. 

Claim 2. ip^ is trivial if k > 5. Moreover, either ^3 or ^4 is nontrivial, but not both. 
Indeed, if ip k is nontrivial, then is nontrivial by Claim 1, thus 

m 

— >h k > mi(.M a _i,i) = N g-i 1 

therefore A; < 4 as m < 5N~_ 1 . If both ^3 and ^4 are nontrivial, then 

m > 3/i 3 + 4/i 4 > 7N~_ X , 

which also contradicts m < 5N g _ 1 . Finally, by Lemma 15.41 one of the ipk with k > 3 must be 
nontrivial, so either ^3 or ^4 is nontrivial. 

Claim 3. ip 3 is nontrivial. 

Suppose otherwise that ^3 is trivial. Then, by Claim 2, -04 is nontrivial, thus h.4 > N~_^. As 
/ii + 4/14 < m < 5N g _ 1 , it follows that hi < Ng~L\, thus acts trivially on fii. So, 

S(# r i,i))c0 4 cs(ff). 

Since 5(</j(A / t g _i,i)) contains S({w%, W2)), which has the same cardinality as S(H), the above 
inclusions are indeed equalities. 

So, all nontrivial orbits of H have length 4. Applying Lemma [2.21 we get that W2 is a product of 
disjoint 3-cycles and, for i € {1, 3}, all nontrivial orbits of (1V2, Wi) have length 4. By Lemma [1.41 
it follows that w% = W3. Hence, we also have 

^1 = (^3^4)^1 (u>3W4) _1 = (W3W4)W3(WSW4 : )~ 1 = W4 , 

thus W3 = W4, therefore Wo = w± = ■ ■ ■ = W2 g - This implies that the image of tp is cyclic, and 
hence, because M. g ,i is perfect, p is trivial - a contradiction. 

Claim 4. ipx is nontrivial. 

Suppose instead that ip\ is trivial. Then we have the inclusions 

S{y{M g -x, x )) d 3 C S(H) 

which imply as in the previous claim that S(H) = Q3. So, all nontrivial orbits of u>2 g and 
{w2 g , W2 g +i) have length 3; thus, by Lemma [L~3"1 W2 g = w^g+i- It follows that the image of ip is 
cyclic, and hence, because M. g ,i is perfect, ip is trivial - a contradiction. 
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The following claim is a direct consequence of the braid relation between W2 g and W2 g +i (see 
the proof of Lemma II. 3|) . 

Claim 5. Sl 3 \, . . . ,03 h 3 are precisely the common 3-orbits of W2 g and u>2 g +i- 

We turn now to conclude the proof of Proposition 15,61 with counting arguments. If (61, ... , b p ) 
is a p-chain of nonseparating closed curves, we denote by c p (bi, . . . ,b p ) the number of common 
3-orbits of tpfT^), . . . , tp(Tb ). We do not assume that the whole chain is nonseparating, but we 
suppose that each pair 6j+i) is nonseparating, so that (Tfe.,Tfe. +1 ) is conjugate to (T2 g ,T2 g +i) 
(see Lemma l2.2p . By Claims 1, 3 and 5 we have 

02(61,62) = c 2 (a 29 ,a2 9 +i) = h 3 > N g _ x , 

and by Claim 4 we have h\ > N~_y_. Recall also that t 3 is the number of 3-orbits of W2 g - 
Note that the supports of the 3-orbits of W2 g that are not included in Q3 are included in X = 
{1, . . . , m} — ($^3 U Oi); thus there are at most \X\ /3 of them. Again by Lemma 12.21 it follows 
that for all i S {2, . . . ,p} among the 3-orbits of <p(Ti) there are at most \X\ /3 that are not 
3-orbits of ^>(Tj_i). We therefore obtain in turn 

I .XI 

03(61, b 2 ,b 3 ) >h 3 — , 04(61,62,63,64) > h 3 — , 

3\X\ 

05(61,62,63,64,65) > h 3 — = h 3 - \X\ . 

Using again the assumption that m < UNg^, we have 

\X\=m-h 1 - 3/i 3 < 4NJ_ X - 3/i 3 < h 3 . 

Hence 

05(61,62,63,64,65) >h 3 -\X\>0. 

So, there is a common 3-cycle in the decomposition of every generator wo, . . . , of (p(M.2,i)- 
By restricting attention to the support of this 3-cycle, we deduce that ip induces a nontrivial 
homomorphism from .M 2,1 to ©3 whose image is generated by elements of order 3; in other words, 
from M21 onto the cyclic group of order 3. This, however, contradicts Proposition 13.21 □ 

5.2 Normal generation of Ker8 g ^ by the square of a Dehn twist 

Thanks to Corollary 15.31 and Propositions 15.5 1 and 15.61 we know (under the inductive hypothesis 
that Theorem 10.31 holds for a surface of genus g — 1) that, if g > 4 and ip : M. g ,\ — > &m 
is a permutation representation with m < 5N~__ l , then Wi = <p(Ti) is an involution for all 
i € {0, 1, . . . , 2g + 1}. So, in order to prove Theorem 15. H still under the inductive hypothesis, it 
suffices to show the following. 

Theorem 5.7. Let g > 2. Then the kernel of 9 9t i is the smallest normal subgroup of M g i 
containing Tf. 

This theorem is known to experts but, as far as we know, is nowhere in the literature. So, we 
include a proof but skip some details. 
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Proof. We say that a normal subgroup H of a group G is normally generated by a subset 
S C H if i7 is the smallest normal subgroup of G that contains 5". For i,j £ {1, ... , g} we 
denote by the g x g matrix whose entries are all zero except the (i, j)-th entry which is equal 



to 1. Then the kernel of fj, g : Sp 2g (Z) — > Sp 2g (F 2 ) is normally generated by 



la 2ei 







where 



I g denotes the identity matrix. This fact can be deduced from [3] (see also [3T] ) and its proof is 
left to the reader. 

We denote by 6 g : M. gj % — > Sp 29 (Z) the natural epimorphism, so that 9 g< i = \i g o 8 g . Consider 
the simple closed curves c 2 , . . . , c g , c' 2 , . . . , c' g , d 2 , . . . , al g illustrated in Figure 5.1. By [27] and [I] 
(see also [28]), Ker6* g is normally generated by {T Ci T7 1 3 Td j | 2 < i < g}. On the other hand, 



a direct calculation shows that g (T^ 



h 2e i,i 



By the above, it follows that Ker 9 g ^\ is 



normally generated by {T Ci T , 1 ,T di | 2 < i < g} U {T- 2 }. 




Figure 5.1. Some curves in "E g i. 



Let H be the normal subgroup of M. g ,i normally generated by T 2 , and let tt : M. g> i — > M 9j i/H 
be the quotient map. The following relations hold in M g \/H: 



1 



if 1 < i < 2g , 
^{Ti)Ti{T 3 ) = n{Tj)ir(Td if \i - j\ > 2 , 

7r(ri)7r(r i+ i)7r(Ti) = ^(T, +1 )^(T,)^(T, +1 ) if 1 < i < 2g - 1 . 

Thus, there exists a homomorphism q : &2g+i — > Mg,x/H that sends Sj = (i, i+ 1) to 7r(Tj) for 
all 1 < i < 2g. On the other hand, the following relations hold in M g ,i (see [2T], for instance): 

T di = (TiT 2 ■ • • T 2i _ 2 ) 4i " 2 , 
T Ci T c , = (T 1 T 2 ---T 2i _ 1 ) 2i , 

, g}. Moreover, since s\s 2 ■ ■ ■ Sj is a cycle of length j + 1, 

(sis 2 --- Sj ) i+1 = 1, forl<j<2 5 . 



whenever ? 6 {2, . 



It follows that 

vr(T d J = g(( aiSa • • • s 2l . 2 f^ 2 ) = q(l) = 1 , 
tt^TJ 1 ) = n(T Ci T c[ ) = q((s lS2 • • • s 2i ^) 2i ) = g(l) = 1 

for all ie{2,...,g}. Thus, # = Ker0 S) i. 



□ 
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Proof of Theorem 15.11 For g > 4, let (p : M g ,i —> & m be a nontrivial homomorphism, 
with m < 5N~_ V As pointed out before, thanks to Corollary 15.31 and Propositions 15.51 and 15.61 
Wi = (f(Ti) is an involution for all i £ {0, 1, . . . , 2g + 1}. By Theorem 15.71 we conclude that there 
exists a homomorphism <p : Sp 2s (F 2 ) &m such that ip = (p o Q g \. □ 

6 Large subgroups of Sp 2ff (F2) 

The aim of this section is to prove Theorem 10.21 which, together with Theorem 15.11 proves 
Theorem 10.31 for the case of a surface with n = 1 boundary component. The extension to 
surfaces with several boundary components will be the object of Section [HJ 

The maximal subgroups of the finite classical groups have been described using the classification 
of finite simple groups. (Below, 2 n denotes an elementary abelian group of rank n, A.B an 
extension of a group A by a group B, and r a cyclic group of order r.) Their orders are also 
known (see for example |30j). 

Theorem 6.1. ([22, Theorem 4.1 and §3]; [21 §1]) If H is a maximal subgroup o/Sp 2s (F2) for 
g > 1, then one of the following holds. 

(1) H = 2 - (F 2 ), of order 2S 2 ~9+ 1 (29 + 1) nf=i(2 2i - 1). 

(2) H = 0+ (F 2 ), of order 2^-9^^ _ 1) nf~i(2 2i - 1). 

(3) \H\ < 2 & 9. 

(4) H < &2 g +2, of order at most (2g + 2)!. 

(5) H = Sp 2 fc(F2>-).r, where r > 1 is a prime divisor of g and kr = g. Here, H has order 

r-2^n-=i(2 2 "-i)- 

(6) H = Sp 2fe (F 2 ) l & r , where r > 1 is a divisor of g and kr = g. Here, H has order 
r!-2^nti(2 2i -l) r . 

(7) H is the stabilizer of a totally isotropic subspace o/F 2 9 under the natural action o/Sp 23 (F 2 ). 
That is, H is 

(2^.2 2fc k-*>) x (Sp 2 ,_ 2fe (F 2 ) x GL fe (F 2 )) 
for some integer k E {1, . . . ,g}, and has order 

292 ■ (nV - !)) • (nV +i - !)) • 

(8) H is the stabilizer of a nonsingular subspace o/F 2 9 under the natural action o/Sp 29 (F 2 ). 
That is, H is Sp 2fc (F 2 ) x Sp 2s _ 2fc (F 2 ) for some k € {1, . . . , g — 1}, and has order 

y+2*»-2 fl * . ^JJ(2 2i - l)j • (j[(2 2 * - l)j . 
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Moreover, the maximal subgroups falling within case (1) lie in a single conjugacy class, as do 
the subgroups falling within case (2). 

Proof. The first part is shown in |22t Theorem 4.1 and §3]. Their orders may be found in [301 
pp. 19, 70, 141]. For the last two cases, the description of H follows from [32] p. 63], for example. 
Cases 2 and 1 correspond to the Aschbacher class Cg [2] §1], that is, contain the subgroups 
of Sp 2g (F 2 ) stabilizing some quadratic form polarizing to the symplectic bilinear form defining 
Sp 2g (F 2 ); Case 2 corresponds to forms of Witt index g, Case 1 to forms of Witt index g — 1. 
By [2] Theorems BA and BO], the action of Sp 29 (F 2 ) is transitive on stabilizers of forms of 
the same similarity type, showing that the groups falling within Case 2, respectively the groups 
falling within Case 1, form a single conjugacy class. □ 

Corollary 6.2. Let g > 3 and let H be a subgroup o/Sp 29 (F 2 ) of index (strictly) less than 2N~ . 
Then either H S 2g (¥ 2 ) or H ^ Oj(F 2 ). 

Remark. S , 23 (F 2 ) is an index 2 subgroup of O^O^) and, therefore, an index 2iV~ subgroup 
ofSp 2g (F 2 ). 

Proof. Recall again that N~ = [Sp 29 (F 2 ) : 2g (F 2 )] and note that the result holds for proper 
subgroups of 0.^(F 2 ) and 0^(F 2 ) because |0 2 4 ff (F 2 )| < |0 29 (F 2 )|. Thus, we need consider 
only maximal subgroups H. For applications, the assertion is presented in terms of the index; 
however, the data relate to the order, so one needs to check that of the maximal subgroups listed 
in Theorem 16.11 in all except for the first two cases the order of H is less than 

|C>2g(F2)l = 2* a -»(2» + 1) JJ(2 a - 1) . 

2 i=i 

For g = 3 the result holds by [8]. (Alternatively, we can compute the conjugacy classes of 
maximal subgroups of Sp 6 (F 2 ) using Magma [5]: their orders are 1512, 4320, 4608, 10752, 
12096, 23040, 40320, 51840. The maximal subgroups of order 40320 and 51840 can be checked 
to be isomorphic to Og (F 2 ) respectively O f r(F 2 ), so the claim holds.) 

So, assume that g > 4. In general, it is a routine matter to use the formulae of Theorem 16.11 
to check that H has order less than |0^(F2)|/2. To indicate the flavor of the verification, we 
discuss the two most delicate cases, namely (6) and (8). 

Case (6). First observe that r! < 2( r2 ~ 2r+3 )/ 2 , and, for r in the range [2, g], the function 
(r 2 — 2r + 3)/2 + — achieves its maximum value of (g 2 + 3) /2 at the endpoints. Thus, on putting 

j = ir, we have that \H\ = rl ■ 2~ Ili=i(^ 2 * — 1S bounded above by 

2 (s 2 +3)/2 Yl (2 2 J-1) < 2^ 2+2 < ?+3 )/ 2 (2S + l) Yl (2 2j -l). 

r\j, j<g r\j, j<g-l 

Since 2 (f 2 + 2 9+ 3 )/ 2 < 22 2 -9(2 2 - 1), and (2 2 - l)U r \j, jKg-l^ ~ l ) < ilf=i(2 M - 1), we obtain 
\H\ < _(F 2 )|/2, as required. 
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Case (8). Since, for k in the range [l,g — lj, the function 2k{k — g) achieves its maximum of 
—2(g — 1) at the endpoints, we have 2 9 +2k ~ 2 9 k < 2 9 -a. Meanwhile, using the symmetry of 
the product below to assume that k < |, and observing that i < i + g — k = j say, gives 

(nV - 1)) • (n^ - !)) 

/g-k \ /k-1 \ 

=(2 2fc - 1) f n( 22i - ) • f - !)) 

/g-k \ / s-i 

<2»+i) n^- 1 ) • n ^-^ 

\i=l / \j=g-k+l 

When combined with the exponential inequality above, this again yields \H\ < |0^(F2)|/2. □ 

Proof of Theorem [E3 As stated before, we have [Sp 2fl (F 2 ) : 0^(F 2 )] = N~ = 2 9 ~ 1 (2 9 - 1) 
and [Sp 29 (F 2 ) : Oj(F 2 )] = N+ = 2 9 ~ 1 (2 9 + 1) by [301 PP-70 and 141]. Finally, Corollary O 
ensures that the conjugacy classes of 0;^(F 2 ) and 2g (F 2 ) are the only conjugacy classes of 
subgroups of Sp 29 (F 2 ) of index less than 2iV~. □ 

Now, thanks to Theorem 15.11 and Theorem 10.21 we can prove our main theorem for the case of 
a surface with n = 1 boundary component. However, we have to keep the inductive hypothesis 
(that Theorem 10 . 3 1 holds for a surface of genus g— 1), as we have used the equality m^Afp-i 3) = 
N~_^ in the proof of Proposition 15.21 

Proposition 6.3. Let g > 4. 

(1) O gl is the unique subgroup of M g ,i of index N~ = 2 9 ~ 1 (2 9 — 1), up to conjugation. 

(2) Og^i is the unique subgroup of M g ,i of index N~£ = 2 9 ~ l (2 9 + 1), up to conjugation. 

(3) All the other proper subgroups of M g ,i are of index at least 5N~ —1 > iV+ . 



Proof. Let H be a subgroup of M g ,i of index m < 5 N g _i. By Theorem 15.11 there is a subgroup 
H of Sp 23 (F 2 ) of index m such that H = g> l(H). Since 5N g _ 1 < 2N~, by Theorem [021 
we must necessarily have either H = 0^(F 2 ) or H = 0^(F 2 ) up to conjugation, thus either 
H = 0~ x or H = up to conjugation. □ 



7 Small symplectic representations of Ai g , n 

The aim of this section is to prove Proposition 17.21 which is the same as Theorem \UA\ except 
that we do not state that the decompositions in (1) are unique. The uniqueness will follow from 
Theorem 10.31 proved in Section O 

The decompositions of 4>^ n in (1) reflect analogous properties of the corresponding representa- 
tions of Sp 2s (F 2 ), which in turn arise in a geometric way; while this result may be more or less 
known to experts, we could not locate a reference and hence establish it in Lemma 17. II 
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We choose a basis {ei,...,e2 g } of V = ¥ 2 9 such that (ej,e2 P +i_j) for i = 1, ...,</ are the 
symplectic pairs for the action of Sp 2s (F2); we write i for 2g + 1 — i to shorten notation. For 
a vector x G V, we denote the components of x with respect to the basis {ei, . . . , e 2£ ,} by 
x%, . . . ,X2 g - We consider Sp 29 _ 2 (F2) as a subgroup of Sp 2s (F2) via the the standard embedding 
defined by 





' 1 


••• 















to i — y 












. 


••• 


1 



with respect to the above basis. 

For g > 2 and e G {±}, let (fig : Sp 2s (F2) — ► ©tvj denote the permutation representation induced 
by the action of Sp 29 (F2) on the right cosets of 0\ g {¥2) (cf. Corollary 16. 2p . 

Lemma 7.1. .Lei g > 3. Then, 

^9 + Isp 29 _ 2 (f 2 ) = (^ 9 + -i) 3 © and 9 "ls P29 _ 2 (F 2 ) = (? s -l) 3 © ^9-1 ' 



Proof. It is well-known (see, for example, [12] or [3Q(. ch. 11]) that 4> g and (f> g arise from the 
action of Sp 2s (F2) on the two orbits of quadratic forms polarizing to the symplectic form (— , — ) 
preserved by Sp 2s (F 2 ). The two orbits consist of the quadratic forms of Witt index g (type +) 
respectively Witt index g — 1 (type — ). 

Let Q denote the set of quadratic forms polarizing to (— , — ). The elements of Q are of the form 



Qb(x) = Xixi + (x, b) 



9 

E 



+ (x, b) for bey. 



The symplectic group Sp 2g (F2) acts on Q via (to ■ Q)(x) := Q(cjx) for to G Sp 29 (F 2 ) and Q G Q. 
In particular, (to ■ Qb)( x ) = Qb"( x ) for some b w G V. It is easy to see that this defines 
another action of Sp 29 (F2) on the set V given by to ■ b := b w . (Note that this action does 
not respect the structure of V as a vector space. In particular, it is different from the natural 
action given by matrix multiplication.) It is easy to verify that the restriction of this action to 
Sp 2g _ 2 (F2) < Sp 2g (F2) can be canonically identified with the analogous action of Sp 29 _ 2 (F2) on 
quadratic forms on F^ -2 . 

Now consider the two orbits Q + and Q~ of quadratic forms of type + respectively type — under 
the action of Sp 2ff (F2). We will show that Q e (e G {±}) splits up into four orbits under the 
action of Sp 29 _ 2 (F2), one for each of the 2 2 possible values of (b\, &2g)> an d that three of these 
orbits have the type e while the remaining orbit has the type — e. 

Consider first the orbit Q + of quadratic forms of type +, that is, of Witt index g, and define 









v y 



/ 1 \ 







V / 









V 1 J 



( 1 \ 

1 




1 

V 1 / 
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The quadratic forms Qb;( x ) for i = 0,1,2,3 have Witt index g, that is, are elements of Q + . 
Moreover, Q+ = UUiQl 9 ■ It 

is easy to see that the restriction of Q\ 3i to F 29 2 is of 
type + (Witt index g — 1) for i = 0, 1,2 and of type — (Witt index g — 2) for % = 3. Hence 

0g ls P29 _ 2 (F 2 ) = &g-i) 3 © 4>g-i as claimed. 

The argument for the orbit Q~ of quadratic forms of type — is analogous. □ 

Proposition 7.2. (1) Let g > 3 and n > 1. TTien (f)~ n : M g , n ®jv~ equivalent to 
an extension of the representation {4>g~-i ra ) 3 © ^g-\ n f rom -M-g-i,n to M. g , n , and <fi+ n : 
Mg t n — > & N + is equivalent to an extension of the representation <f>g_i n © (^-i n ) 3 f rom 

Mg-l, n to M g>n . 

(2) Let g > 3 and n > 0. Let b be a nonseparating simple closed curve on Sg m and let Tt, be 
the Dehn twist around b. Then the cycle structure of the image of T^ under <j>~ is 

and the cycle structure of the image of T& under is 

(1) 229_2 (2) 29_2(29_1+1) 



Proof. We first consider Part (1). For a simple closed curve c on we denote by [c] the 
class of c in Hi(T, g> i, Z). We consider the curves «i, . . . , u g , v±, . . . , v g indicated in Figure 7.1, 

, [v g ] as basis elements for iTi(£ fli i,Z). With respect 



and choose [u g \, [it s _i], . . . , [ui], [t>i], 
to this ordering, the bilinear form (- 
the matrix 



V 2 



-) yielding the algebraic intersection number is given by 








M = 



1 

1 




1 









Figure 7.1. Basis for fTi(E ff> i,Z). 



The image of the Dehn twist T^ about a simple closed curve b under the epimorphism 9 9) i : 
M 9t i — > Sp 29 (F 2 ) is the map 

[a] i-> [a] + (a, b) [b] . 

Since M. g -\,i < M g ,i is generated by To, T±, . . . , T 29 _ 2 and since ao, ai, . . . , a 2fl -2 intersect 
neither u g nor v g , it is clear from the above that g> i(M g -i t i) = Sp 29 _ 2 (F 2 ) < Sp 2s (F 2 ), and 
the restriction of to M. g -i,i coincides with 9 g -i,i : — >■ Sp 2ff „ 2 (F 2 ) < Sp 2fl (F 2 ). 
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Let n > 1. Gluing disks along all boundary components of S 9jn but one, we obtain an embedding 
> S 9) i that induces a surjective homomorphism [i g , n : A^ 9i n -» -M^i (see More 
precisely, the epimorphism /Ug^ is defined sending Tj to Ti for all j € {1, ... ,n}, and Tj to Tj 
for all i € {0, 2, ... , 2g}. Observe that 9 g ^ n = 9 g> \ o // g n and the following diagram commutes 



M 9-l,n 

f^g — l,n 

Mg-1,1- 



M 



M 



9,11 

f-^g.n 

9,1 



where M. g _i n — > M. g ^ n and M. g -i,i — >■ are the natural embeddings described in Section [2l 

By the above, it follows that O g , n (-M-g-i,n) = Sp 29 „ 2 (F 2 ) < Sp 2ff (F 2 ), and the restriction of 9 g ^ n 
to M g -i, n coincides with 9 g -i, n ■ M g -^ n -)• Sp 29 _ 2 (F 2 ) < Sp 29 (F 2 ). 

We have 

^9 ls P2g _ 2 (F 2 ) = (^-l) 3 ©^-l alld ^ls P29 _ 2 (F 2 ) 

from Lemma 17. 11 By composing with 9 gn we conclude that 



9,n 



\M 



9-1, n 



■'g-l,n 



and 



g,n 



M a - 



9—1," 



(ft 



Part (2) for n > 1 follows by induction on g, using the cycle structures (2) (1) of <j) 3 n (Tb) and 
(2) 10 (l) 16 of for 5 = 3 (see Lemma HID, and the equivalence ^g, n |>t s _ ljn = (i g -i, n ) 3 

^g-l i n proved above for g > 4. 

Gluing a disk along the boundary component of E 9) i we obtain an embedding E 9) i E 9i o that 

A^ 9i o (see e.g. [26J). Observe that 9 9t i = 9 g $ov. 



induces a surjective homomorphism v : -M 9) i 
Thus, if 6 is a nonseparating simple closed curve in then Sj i(?&) 

(p e gl (Tb) = (j>g (Tfj). This proves Part (2) for n = 0. 



9g,o(Tb)', therefore 
□ 



8 Surfaces with multiple boundary components 

We start the section with a geometrical interpretation of the decomposition given in Proposi- 
tion El 

Take e G {±} and consider the representation <\f g 1 : .M fll i — >• ©jvj. Let m; denote the image of 
Ti under cp e gl for all i G {0, 1, . . . , 2g + 1}. Set # = (il>2 9 , iU2g+l)- Since u?j has order 2, there is 
an epimorphism from the group 

H = (xo,Xi | Xq = x\ = 1, XoXiXo = XiXo^l) = 63 

to H which sends Xi to W2 g +i for z G {0, 1}. In particular, the order of H divides 6. 

For > 1, let fife denote the union of the fc-orbits f^, of H, 1 < i < h^. Since the image 
= ( w 0i • • • i w 2g-2) belongs to the centralizer of H, it acts on the set of orbits 
{fifc i 3 • • • ,flfc h fc } as well as on the whole set £lk- The first action induces a homomorphism 
i>k : J\A g -i^i — > &h k and the second a homomorphism Vfc ■ -Mg-1,1 — > &kh k - The following 
lemma describes these representations. 
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Lemma 8.1. Let g > 3. With the above notation, the following hold. 

(1) h 3 = N*_ v hi = N~l v and h k = for all k {1, 3}. 

(2) v 3 c/> e g _ hl , ^ 3 = (<^-i,i) 3 , and = Vi = 

(3) {l,...,N g } = S((f) € gl (M.3^)), where, for G < &n^, S(G) denotes the support of G, and 
•M 3j i is the subgroup of M. g i generated by Tq,Ti, . . . ,Tq. 

Proof. We prove by induction on g that {wi,W2) has N g _± 3-orbits, has no other nontrivial 
orbit, and has N~*^ fixed letters. The case g = 3 follows from Lemma 14, 1\ and the inductive 

step can easily be derived from the formula <j> g> i\M g -i^ — (0g-i,i) 3 © ^g-i l °^ Proposition 17.21 
Thanks to Lemma 12.21 this proves Part (1). 

Let be a 3-orbit of H. We may write = {a,, bi, c{\ so that the restriction of W2 g to 
is the transposition (ai bi) and the restriction of W2 g +i is (6, q). Set = — S^y^g+i) = 
K 1 < i < JV^_i}, «| = ^3 - S(w 2g ) = {a; l<i< N^}, and «| = n 3 - U «1) = {&;; 1 < 
i < A r |_ 1 }. Then is invariant under the action of Ai g -i t i for £ = 1,2,3, and the action of 
on fi| is equivalent to z/3. So, 

<#?,lK-i,i = V'S © V>1 = (^3) 3 © ^1 • 

For the case g = 3, one can easily check the equivalences v^ = ^>\i an d v\ — <j>2i us hig Lemma [3. II 
and Lemma so Part (2) holds in this case and we can assume g > 4. Recall that /13 = N g-1 
and hi = N g ^. By Lemma 14.11 (if g = 4), respectively Proposition 16.31 (if g > 4), 1/3 and z^i are 

either trivial or of the form v 3 = ! © 1 9 , respectively 1/1 = 1 © lg'- Now, the formula 
of Proposition 17.21 implies that ^ i|.M 9 _i 1 has no fixed letter, and thus both 1/3 and vi must be 
nontrivial and we must have q = q' = 0, \x = e, and // = — e. So Part (2) holds. 

Part (3) follows by induction on g using Lemma 14.11 for the case g = 3 and the equivalence 
0g,ilx 9 _i,i — (^_i,i) 3 © for the inductive step. □ 

Now, we finish the proof of Theorem 10.31 with the following. 
Proposition 8.2. Let g > 4 and n > 0. 

('ij * s ^ e unique subgroup of M g , n of index N~ = 2 9 ~ 1 (2 9 — 1), up to conjugation. 

(2) 0^ n is the unique subgroup of \M. g>n of index iV+ = 2 9_1 (2 9 + 1), up to conjugation. 

(3) All the other subgroups of M g , n are of index at least 5N g _ 1 > . 

Proof. Recall that we are under the inductive hypothesis stated at the beginning of Part m 
that is, Theorem 10.31 holds for a surface of genus g — 1. 

The case n = 1 is proved in Proposition 16.31 and the case n = follows from Proposition 16.31 
Theorem 10.21 and the existence of the epimorphisms M.g,i -» -Mgfi, -M.g,o Sp 29 (F2) described 
in Section So, we may assume n > 2. 
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Let (p : M g ,n —> 6 m be a nontrivial transitive homomorphism with m < 5N g _ 1 . As ever, 
for i G {0, 2, . . . , 2g + 1} and j G {1, . . . , n}, we denote by and bj the simple closed curves 
illustrated in Figure 2.2, we denote by Tj the Dehn twist about aj, by T'- the Dehn twist 
about bj, and we set Wi = tp(Ti) and w'j = <p(Tj). For j G {l,...,n}, we denote by E^ 

a tubular neighborhood of bj U ao U (U^£ 2 a «)- Observe that E^ is a subsurface of E 9)n of 
genus 5 with a unique boundary component, and the inclusion ^ E gjra induces an injective 
homomorphism jj : M g ,i — > M g , n which sends T\ to Tj and Tj to Tj for all i G {0, 2, ... , 2g]. 
Set ifj = ip o jj : Mg t i — > <3 m . By Proposition I6.3| (fj is of the form tpj = ipj © l qj where ipj is 
conjugate to an element of {^,^7^ and l gj : .M ff! i — > & qj is the trivial representation. 
Let E' be a tubular neighborhood of uf =0 a,2 g -i- Then E' is a subsurface of E 9j „ of genus 3 with 
a unique boundary component, and is included in E^') for all j G {l,...,n}. Moreover, the 
inclusion E' <— >■ E 9in induces an embedding 7' : A^3,i — >■ -M g ,n- Set ip' = ip o >y' : A^3,i — >■ © m . 
We have = 5(^'(Ai 3 ,l)) by Lemma 0(3) for all j, and \Jj =l S(<pj(Mg t i)) = 

S(<p(M g , n )), thus 

S(<p(M g>n )) = S(ipj(M g ,i)) = S(ip'(M 3<1 )) . 

Since ip is transitive, it follows that there exists e G {=t} such that m = N* and yjj is conjugate 
to (f>g 1 for all j G {1, . . . , n}. 

Set = {u)2 g , W2g+i) ; note that C ("|. 7j(A^ g ,i). For k > 1 we denote by 0^ the union 
of the fc-orbits Qk,i of H, 1 < i < h^. On the other hand, we assume that M g —i >n is the 
subgroup of M.g, n generated by T[, . . . , T^,Tq, T2, . . . , T^g-2- Since the image ip(Ai g -i in ) lies in 
the centralizer of H, it acts on the set of orbits {f^i, . . . ,&k,h k } as well as on the whole set 
f2fc. The first action induces a homomorphism i/j. : A4 s _i ira — > and the second induces a 
homomorphism tpf. : M. g -i t n — > &kh k - Applying Lemma IBTTl fl) to f>j{M- g ,i) for any j, we get 
that h 3 = A^„ 1; hi = N g ^, and h k = if k {1, 3}. 

As in the proof of Lemma [8.11 we set = — S(w2 g +i), = — S(w2 g ), and fi| = 
O3 — (f2g U 0|). Then is invariant under the action of A4 5 _i iW for £ = 1,2, 3, and the action 
of Mg—i^n on O3 is equivalent to ^3. Hence, 

v\M g - lin = i>3 © *h = (^3) 3 ©^i- 

Since h%,hi < N£_ 1} ^y induction we have ^(Tj) = f3(T{) and fi(T-) = ^(T^), thus ^(^j) = 
<p(T{) for all j G {1, • • ■ , n}. Since 971 is conjugate to ^ x , we conclude that is conjugate to 

n,n- □ 

Proof of Theorem 10.41 This follows from Proposition 17.21 and Theorem [031 proved above. □ 

References 

[1] J. Aramayona, J. Souto. Homomorphisms between mapping class groups. Preprint. 
larXiv:1011.1855l 

[2] M. Aschbacher. On the maximal subgroups of the finite classical groups. Invent. Math. 76 (1984), 
no. 3, 469-514. 

[3] H. Bass, J. Milnor, J.-P. Serre. Solution of the congruence subgroup problem for SL„ (n > 3) 
and Sp 2 „ (n>2). Inst. Hautes Etudes Sci. Publ. Math. 33 (1967), 59-137. 



Finite index subgroups of mapping class groups January 20, 2013 



28 



[4] J. S. Birman. On Siegel's modular group. Math. Ann. 191 (1971), 59-68. 

[5] W. Bosma, J. Cannon, C. Playoust. The Magma algebra system I: The user language. J. 
Symb. Comput. 24 (1997), 235-265, |htt p :/ /magma . m at hs.usyd.edu.au/magma 

[6] M. R. Bridson. Semisimple actions of mapping class groups on CAT(O) spaces. Geometry of Rie- 
mann surfaces, 1-14, London Math. Soc. Lecture Note Ser., 368, Cambridge Univ. Press, Cambridge, 
2010. 

[7] M. R. Bridson. On the dimension of CAT(O) spaces where mapping class groups act. J. Reine 
Angew. Math., to appear. iarXiv:0908.0690l 

[8] J. H. Conway, R. T. Curtis, S. P. Norton, R. A. Parker, R. A. Wilson. Atlas of finite groups. 
Maximal subgroups and ordinary characters for simple groups. With computational assistance from 
J. G. Thackray. Oxford University Press, Eynsham, 1985. 

[9] B. N. Cooperstein. Minimal degree for a permutation representation of a classical group. Israel J. 
Math. 30 (1978), no. 3, 213-235. 

[10] H. S. M. Coxeter, W. O. J. Moser. Generators and relations for discrete groups. Springer- Verlag, 
Berlin-Gottingen- Heidelberg, 1957. 

[11] L. E. Dickson. Representations of the general symmetric group as linear groups infinite and infinite 
fields. Trans. Amer. Math. Soc. 9 (1908), 121-148. 

[12] R. H. Dye. Interrelations of symplectic and orthogonal groups in characteristic two. J. Algebra 59 
(1979), no. 1, 202-221. 

[13] B. Farb. Some problems on mapping class groups and moduli space. Problems on mapping class 
groups and related topics, 11-55, Proc. Sympos. Pure Math., 74, Amer. Math. Soc, Providence, RI, 
2006. 

[14] B. Farb, D. Margalit. A Primer on Mapping Class Groups. Princeton University Press, to appear. 

[15] L. Funar. Two questions on mappping class groups. Proc. Amer. Math. Soc. 139 (2011), no. 1, 
375-382. 

[16] E. K. Grossman. On the residual finiteness of certain mapping class groups. J. London Math. Soc. 
(2) 9 (1974/75), 160-164. 

[17] J. L. Harer. Stability of the homology of the mapping class groups of orientable surfaces. Ann. of 
Math. (2) 121 (1985), no. 2, 215-249. 

[18] A. Hurwitz. Uber algebraische Gebilde mit eindeutigen Transformationen in sich. Math. Ann. 41 
(1893), 403-442. 

[19] S.P. Kerckhoff. The Nielsen realization problem. Ann. of Math. (2) 117 (1983), no. 2, 235-265. 

[20] M. Korkmaz. Low- dimensional homology groups of mapping class groups: a survey. Turkish J. 
Math. 26 (2002), no. 1, 101-114. 



Finite index subgroups of mapping class groups January 20, 2013 



29 



[21] C. Labruere, L. Paris. Presentations for the punctured mapping class groups in terms of Artin 
groups. Algebr. Geom. Topol. 1 (2001), 73-114. 

[22] M. W. Liebeck. On the orders of maximal subgroups of the finite classical groups. Proc. London 
Math. Soc. (3) 50 (1985), no. 3, 426-446. 

[23] M. Matsumoto. A presentation of mapping class groups in terms of Artin groups and geometric 
monodromy of singularities. Math. Ann. 316 (2000), no. 3, 401-418. 

[24] D. Mumford. Abelian quotients of the Teichmiiller modular group. J. Analyse Math. 18 (1967), 
227-244. 

[25] L. Paris. Small index subgroups of the mapping class group. J. Group Theory 13 (2010), no. 4, 
613-618. 

[26] L. Paris, D. Rolfsen. Geometric subgroups of mapping class groups. J. Reine Angew. Math. 521 
(2000), 47-83. 

[27] J. Powell. Two theorems on the mapping class group of a surface. Proc. Amer. Math. Soc. 68 
(1978), no. 3, 347-350. 

[28] A. Putman. Cutting and pasting in the Torelli group. Geom. Topol. 11 (2007), 829-865. 

[29] C. C. Sims. Computation with finitely presented groups. Encyclopedia of Mathematics and its 
Applications, 48. Cambridge University Press, Cambridge, 1994. 

[30] D. E. Taylor. The geometry of the classical groups. Sigma Series in Pure Mathematics, 9. Helder- 
mann Verlag, Berlin, 1992. 

[31] J. Tits. Systemes generateurs de groupes de congruence. C. R. Acad. Sci. Paris Ser. A-B 283 (1976), 
no. 9, Ai, A693-A695. 

[32] R. A. Wilson. The finite simple groups. Graduate Texts in Mathematics, 251. Springer- Verlag 
London, Ltd., London, 2009. 

[33] B. P. Zimmermann. A note on minimal finite quotients of mapping class groups. Preprint. 
larXiv:0803.3144l 

A. Jon Berrick, 

Department of Mathematics, National University of 

Singapore, 10 Lower Kent Ridge Road, Singapore 119076, SINGAPORE. 

E-mail: berrick@math.nus.edu.sg 
Volker Gebhardt, 

School of Computing & Mathematics, University of Western Sydney, Locked Bag 1797, Penrith 
NSW 2751, AUSTRALIA. 

E-mail: v.gebhardt@uws.edu.au 



Finite index subgroups of mapping class groups January 20, 2013 



30 



Luis Paris, 

Universite de Bourgogne, Institut de Mathematiques de Bourgogne, UMR 5584 du CNRS, B.P. 
47870, 21078 Dijon cedex, FRANCE. 

E-mail: lparis@u-bourgogne.fr 



